Rethinking Crowdsourcing Annotation: Partial Annotation With Salient Labels for Multilabel Aerial Image Classification

Annotated images are required for both supervised model training and evaluation in image classification. Manually annotating is arduous expensive, especially multi-labeled images. A recent trend conducting such laboursome annotation tasks through crowdsourcing, where annotated by volunteers or paid workers online (e.g., of Amazon Mechanical Turk) from scratch. However, the quality crowdsourcing annotations cannot be guaranteed, incompleteness incorrectness two major concerns annotations. To address concerns, we have a rethinking annotations: Our simple hypothesis that if annotators only partially annotate multi-label with salient labels they confident in, there will fewer errors spend less time on uncertain labels. As pleasant surprise, same budget, show classifier can outperform models fully method contributions 2-fold: An active learning way proposed to acquire images; novel Adaptive Temperature Associated Model (ATAM) specifically using partial We conduct experiments practical data, Open Street Map (OSM) dataset benchmark COCO 2014. When compared state-of-the-art classification methods trained images, ATAM achieve higher accuracy. The idea promising data annotation. code publicly available.